Solving Inverse Problems Using an Em Approach to Density Estimation

This paper proposes density estimation as a feasible approach to the wide class of learning problems where traditional function approximation methods fail. These problems generally involve learning the inverse of causal systems, speciically when the inverse is a non-convex mapping. We demonstrate the approach through three case studies: the inverse kinematics of a three-joint planar arm, the acoustics of a four-tube articulatory model, and the localization of multiple objects from sensor data. The learning algorithm presented diiers from regression-based algorithms in that no distinction is made between input and output variables; the joint density is estimated via the EM algorithm and can be used to represent any input/output map by forming the conditional density of the output given the input. Causality in physical systems induces directionality in the relations between variables measured from them. Thus, one can generally deene a forward and an inverse direction of mapping. The forward direction is the causal direction, for example, from the forces applied to an object to the motion outcome, from the joint angles of an arm to the Cartesian coordinate of the nger, or from the connguration of a vocal tract to the sound frequencies produced. Similarly, the inverse direction is the non-causal direction. If the goal is to control the physical system the the inverse direction of mapping is particularly relevant. Returning to the above examples, this is the mapping from desired motion of an object to the forces required, from desired Cartesian nger coordinates to required joint angles, or from desired sound frequencies to required vocal tract connguration. In general the forward direction will be a function, whereas the inverse direction may be one-to-many and therefore not a function. One-to-many relations are often diicult to learn with function approximation methods. This diiculty arises from the fact that if the image of an input is a non-convex region in the output, then the least-squares solution may fall outside this region (for further discussion of non-convexity see 12]). This paper proposes density estimation as a feasible approach to the wide class of non-convex learning problems where function approximation and non-linear regression methods fail. The learning algorithm presented here diiers from regression-based algorithms in that no distinction is made between input and output variables; the joint density is estimated and this estimate can then be used to form any input/output map. Thus, to estimate the vector function y = f(x) the joint density …